🟡 MEDIUM data_stream/ami/.../default.yml:12

New CEF/UDP processing path ships without pipeline tests

This PR adds a brand-new CEF processing branch here (- pipeline: guarded by if: ctx.cef != null) that calls the new 730-line cef-pipeline.yml. None of it is exercised by a test: the only pipeline fixture, _dev/test/pipeline/test-ami.json, contains JSON-input events only, so every rename in cef-pipeline.yml and the entire cef != null branch run untested. This is a newly introduced code path (not a pre-existing gap), so a regression in CEF field renaming, the cef.extensions -> gigamon.ami mapping, or the new TcpRtt* converts would not be caught.

Recommendation:

Add a CEF pipeline test fixture and expected output covering the cef != null branch, e.g. _dev/test/pipeline/test-cef.log with a matching test-cef.log-expected.json. Configure the CEF test in _dev/test/pipeline/test-common-config.yml so the decode happens correctly:

fields:
  tags:
    - preserve_original_event
dynamic_fields:
  "event.ingested": ".*"

The fixture should include at least one raw CEF message that populates gigamon.ami.* extension fields (src/dst IP, ports, the new TcpRtt* and app_tags fields) so the new pipeline and converts are validated.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

I wanted to confirm the right approach, as I hit a constraint while implementing it.

The recommendation was to add a raw test-cef.log fixture and configure decoding in test-common-config.yml. However, in this package CEF decoding is done by the agent-side decode_cef processor (in udp.yml.hbs), and decode_cef is not available as an Elasticsearch ingest processor. When I tried adding decode_cef to the ingest pipeline (default.yml), Put Pipeline fails with:

No processor type exists with name [decode_cef]

Because pipeline tests only exercise the ingest pipeline (not agent processors), a raw test-cef.log never gets decoded during the test, so ctx.cef stays null and the cef != null branch isn’t actually exercised (the generated expected doc only contains the raw message and no gigamon.ami.* fields).

To genuinely cover the branch, I added a pre-decoded CEF fixture (test-cef.json) that mirrors the agent’s decode_cef output (populated cef.extensions), which correctly triggers the cef != null path and validates the cef.extensions → gigamon.ami renames, the new TcpRtt* converts, and app_tags. Both tests pass:

test-ami.json PASS
test-cef.json PASS

Could you please confirm this approach is acceptable, i.e. using a pre-decoded test-cef.json fixture instead of a raw test-cef.log, given that decode_cef can only run agent-side? If you’d prefer a different pattern for covering the CEF branch, happy to adjust.